Thanks for stopping by
our bulletin board.
Please take just a moment to register so you can post your own questions
and reply to topics. It is free and takes only a minute to register. Just click on the register
link
posted 05-03-2008 09:23 PM
It is a fundamental principle of scientific testing that all measurements are estimates, and that all estimates include components of error, just as it is a principle of marksmanship that two bullets fire from the same rifle may not penetrate a target at the exact same location. Sighting a rifle, involves putting a few bullets on a target and then making any necessary adjustments to things like elevation and windage after reviewing the pattern on the target. This practical example is instructive because it illustrates the sometimes interactive nature of an independent variable (rounds) and dependent variable (pattern). While the rifle is the independent variable (stimulus), we sometimes evaluate the dependent variable (response) and then make adjustments to the independent variable (sight adjustment). Ideally we put several bullets on the target before making any adjustments. To do otherwise is to sometimes cause a whole lotta confusion, because we have not yet had the chance to evaluate whether an off-center round is a feature of a sight adjustment or simply random error. Multiple rounds on the target solves this problem, because it allows us to achieve a more stable assessment of our centroid.
One difference between hunting and target shooting is that hunting is more like field polygraph, in that we do not evaluate our target for the purpose of providing feedback to the stimulus. Instead we evaluate the target to see whether the target is down or not down, and whether we need to put more rounds (more stimulus) on it to put it down.
Backster, to me, seems to violate this fundamental principle of science when he suggests adjusting the intensity of the RQs or CQs during the middle of a test. It is, in fact, confusing the dependent and independent variables.
Another fundamental principle of measurement is that all measurements are a combination of three factors: 1) the data we obtain, 2) the actual value which we are seeking, and 3) random measurement error. For example, if we are conducting a neuropsychological evaluation on a head injured commercial airline pilot (just the kind of pilot you want on your next flight), we might use a hand-dynamometer from Lafayette Instrument to measure grip strength. Our assessment of grip strength is a feature of 1) actual grip strength, 2) grip strength demonstrated during the test, and 3) the accuracy of the hand-dynamometer. Obviously, there are human factors, such as effort, involved in this. However, the idea of taking multiple measurements to reduce error is so fundamental that even construction worker who frame houses for a living are taught things like measure twice, cut once. What is important is that we take the measurement in the same manner each time. It would make no sense to swith hand-dynamometers to a device with a different shaped grip before taking a subsequent measurement. Neither would it make sense to measure a 2x4 with a measuring tape one time and using eye-ball estimation the second time. In fact, it would probably not be completely satisfying for our construction working to take the first measurement in inches and the second in metric units, simply because it is more difficult to evaluate the stability of the measurement. The point is do it the same way every time. The same goes for target shooting. Same for polygraph testing.
I recently learned that the Marcy Technique (a variant of the GQT and MGQT), involves some serious departures from these scientific principles. Marcy evidently teaches a technique which involves the presentation of the test stimuli three different times (which is correct), but advocates or requires the stimuli be presented differently each time. The first chart is silent-answer. The second is a stim test. The third chart is standard yes/no, and the Fourth chart is a yes-answer test. The result of all this variation, is that there are now four (4) not three (3) dimensions to the variability in our test data: 1) the data we obtain, 2) the actual value which we are seeking, 3) random measurement error, and 4) variation in stimulus presentation. Marcy may be a great guy, and may have made some smart contributions, but this seems to me to be a violation of good scientific testing principles (take multiple measurements to achieve a more stable estimate, and take your measurements the same way every time).
Not to pick on just Marcy or Cleve, but Nate Gordon (in addition to mathematical errors that cannot be reconciled with either simple algebra or inferential statistics) commits similar violations of fundamental testing principles.
The goal of measurement is not to simply produce greater reactions or reactions further from zero, but to produce the most stable estimate of differential reactivity to RQs and CQs. What Nate Gordon doesn't understand is that it is the variance which allows us to produce statistical classifiers (estimates). The goal is that the observed variance is natural variance and not variance induced by the examiner. Just as we want the examinee reacting to the stimuli and not the examiner, we want the examinee to have the opportunity to react to the same stimuli during each test chart. Varying the order of the test questions does not violate this opportunity or requirement. However there is evidence that the question sequence Green Red or Red Green will make a difference in the observed scores. Changing the sequence in the last chart, from Green Red to Red Green, as Gordon does, changes the way we achieve this measurement and can only induce arbitrary variance into the test data. Silent and Yes answered test should be in addition to the collection of three normal test sequences.
Return to our hunting metaphor to illustrate further. If there is a best way to hold a rifle, by design and by experience (experiment), then we should train people to hold it that same way every time. Same grip. same trigger control. Same sight picture on the target. That way we know what happens, and we know how to make any necessary adjustments.
There is no sense in turning the rifle upside down one time, just to see what happens. That is a child's version of a scientific experiment. Good science is also a matter of good principles. Furthermore, if someone wishes to assert that rifle-upside-down mode is actually more effective, the correct procedure would then be to hold the rifle upside down every time we fire it. Doing otherwise distorts the variance of our data, and prevents us from the ability to achieve any reliable statistical classifier.
So, the principles here are 1) take multiple measurements to achieve a stable estimate, and 2) take the measurements the same way/best way every time. This way we can eventually move beyond polygraph techniques that cannot be reconciled with good math and good science. Sure the techniques in question may work adequate, and may even work well. But we all know that the polygraph is not perfect, and can be improved upon. If we hope to hold perceptions of respect and credibility from other sciences, it only makes sense to begin to align our testing and and scoring techniques with the principles of good scientific testing.
.02
(end of rant)
r
------------------ "Gentlemen, you can't fight in here. This is the war room." --(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)
posted 05-04-2008 07:57 AM
Ray, You make excellent points honing in on standardization as a key to reducing the examiner's influence on variability. This is the very reason we wrote the DLST paper and made the suggestion it be considered for screening.
As it appears it will be some time before it can be published, perhaps we should allow others in this forum access for input.
posted 05-04-2008 08:08 AM
Nice analogy Ray! I think not only should we not fire the rifle upside down, we need a newly designed lock, stock, and barrel.
Funny yu should mention L. Marcy---as some years back upon hearing his lecture on his testing format---I was reminded of Vassini's "battle of wits" and made such remarks on this board. Talk about psychologyzing! Marcy's (and to certain extent the Matte Track(s)) all remind me of a Caper. If we apply the Second Law of Thermodynamics to polygraph, than it should become simpler, not more convoluted----and that the tipping point of it's most workable form as it currently exists is pretty much hovering around the best it can do, sans better computer scoring recipes.(IMO).
Why should a polygraph test be a caper?
answer: It shouldn't have to be.
[This message has been edited by stat (edited 05-04-2008).]
posted 05-04-2008 08:11 AM
I say we coin a new term here and now.
Polygraph Caperism; The act of making grand assumptions, bait and switch, and other elaborate distractions in an effort to surreptitiously gather data. aka, a polygraph data heist.
[This message has been edited by stat (edited 05-04-2008).]
posted 05-04-2008 01:30 PM
I think its no secret that I don't generally shy too much from complexity, but lordy some of the complex projective psychologized mental gymnastics we examiners come up with in the name of science...
I heard Nate Gordon talk at APA last year about protecting the first RQ with a non-current exclusive comparison or somethingorother. Seems to me that four studies have suggested time-bars don't do much.
There are plenty of unavoidable complexities, like what happens to our decision alpha when we conduct multiple comparisons (a simple law of probabilities). We don't need to keep making a big deal out of things that aren't really important. But if the data say the question sequence is important, then don't jack-up the question sequence.
All these fancy things, are polygraph capers.
Fancy tests: Yes-B, last word, Rs only, rifle upside down, (BTW, I always prefer to hold a hand-gun side-ways, like a pimp, because it creates a confusing juxtaposition for the enemy who also notices my plastic pocket-protector - the confusion gives me a tactical advantage that would be otherwise unrealized).
And fancy questions: inside track, outside track, left-foot in, left foot out, do the hokey-pokey and shake yourself about, or things like "are you purposefully withholding any memory of information about whether so-and-so raped that other person?"
Fancy tests.
Fancy questions.
Its all too much for me.
I prefer a simple testing approach - single issue Zone exams, multi-facet MGQT exams, and good-ole-fashioned questions like "did you do it."
Then lets put our excess mental effort into the real issues, like developing a more complete and integrated understanding of the range of psychological and physiological constructs which play a role in polygraph testing, and incorporating responsible and accurate mathematical and decision models in our scoring systems, improving interrater consistency by optimizing/simplifying feature definitions. (oops, I forgot, the Matte Quadri-Track Zone test already has perfect 1.0 interrater reliability - probably not).
.02
r
------------------ "Gentlemen, you can't fight in here. This is the war room." --(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)
in your original post, you wrote, "Backster, to me, seems to violate this fundamental principle of science when he suggests adjusting the intensity of the RQs or CQs during the middle of a test."
Now I'm not a Backster graduate so you'll have to tolerate my ignorance. But, exactly how does he recommend "adjusting" intensity of either RQ's or CQ's? Sounds to me like question stomping?!
Please set the record straight if I am wrong about this.
I recall learning more than once that Backster taught to adjust the intensity of the comparison or relevant questions, after evaluating, say, the first test, in which the RQs and CQs may cause reactions of equal intensity. Backster, I think, considered one of the two questions to be defective or not working properly and makes adjustment to the question stimuli at such time.
Is that incorrect?
At the onset of the Matte Python Holy Grail Report, Mangan et al (2008) wrote:
quote:The Quadri-Track Zone Comparison Technique employs the Backster Zone Comparison Technique's basic test structure and quantification system
See the second paragraph on page 2. So, I assume there are a lot of shared ideas and practices between the Matte and Backster techniques. It seems to me that Backster doesn't use, but does not object to, the inside track with his technique.
Matte, on page 281 of his 1996 book wrote:
quote:After the administration of each test chart, the FP conducts a Spot Analysis referring the the Matt 23-Reaction Combination Guide to determine whether all tracks are functioning as designed and apply remedial action when necessary prior to the administration of the next test chart. This may necessitate an increase or decrease in the intensity or strength of one or both control questions, and perhaps a change in the type of control question.
All of these remedies are included in aforementioned [sic.] Reaction Combination Guide. Backster provides an 8-Reaction Combination Guide which he initially developed.
So, Matte clearly advocates this unscientific role reversal of the independent and dependent variables - in which he adjusts the independent variable mid-test if he is not happy with the way the data (dependent variable) are developing.
Matte also seems to attribute this idea to Backster, but I understand that sometimes training evolves with the times, and this may not still be taught.
Do people do this?
On page 292 of Matte's (1996) book, he wrote:
quote:... Furthermore, the FP is responsible for conducting a spot analysis of the physsiological data immeiately after the conduct of each chart to determine whether each type of test question is functioning as deemed necessary, and identify and counter any examinee countermeasures before administering the next polygraph chart in the same test.
I have no real problem with the part about identifying and countering any examinee countermeasures before continuing, but why would that be necessary if the inside track makes the examination immune to countermeasures. Wouldn't the test simply roll right over the CM's like a well-sprung Jeep goes over boulders? If the inside track worked as advertised, the test would simply score to deception without any mid-test concern or adjustment.
On page 325, Matte (1996) wrote:
quote:An analysis is made after each test (chart) is conducted on each Track and/or Spot, using the Matte or Backster Reaction Combination Guide as appropriate, to determine if each type of test question is functioning as designed and whether remedial action is needed prior to the conduc`t of the next test (chart).
(breathe) That is a run-on sentence to rival one of my one. Warms the heart, dunnit.
Anyway, it does seem like Matte attributes in-test question adjustments to Backster.
The Matte Quadri-Track Reaction Combination Guide can be found in Matte's book in a 5 1/2 page table (pages 332 to 337).
Something tell's me that a lot of people may not actually pull this table out in the middle of a polygraph test. Also most examiners I know would be very reluctant to the idea of changing a question mid-test, because the test would never pass a QC review if necessary.
.02
r
------------------ "Gentlemen, you can't fight in here. This is the war room." --(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)
[This message has been edited by rnelson (edited 05-05-2008).]
posted 05-05-2008 01:45 PM
Here's what Backster's "Tri-zone Reaction Combination 'D'" says (under the subheading of "Disorderly flow of psychological set 'lean'* toward deception ?d"):
INDICATION:
"Presence of response to one or both green zone questions in addition to red zone question indicates serious green zone question defect."
REMEDY:
"Reduce intensity of green zone questions by altering subject age categories or changing scope of green zone questions."
[This message has been edited by Barry C (edited 05-05-2008).]
I've noticed a "small" addition to your posting and want to bring it to you attention. Is it really necessary to post a pic of your "privates" on this board, every time you put your name?
posted 05-06-2008 12:28 PM
LOL---Jim--better see someone about those feelings!
I see a man on a chopper riding at me and he has those frilly frays on the ends of his handlebars------which--- I suppose is no less homoerotic than Jim seeing Ray's johnson.
Rorschach---wasn't he the guy in "Welcome Back Kotter" with the dumb laugh?
[This message has been edited by stat (edited 05-06-2008).]
Polygraph Place Bulletin Board
posted 05-03-2008 09:23 PM
posted 05-05-2008 12:01 PM
posted 05-06-2008 12:14 PM
